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Abstract 


High-impact practices, such as learning communities, capstones, undergraduate research, and community-based 
experiences, are effective pedagogies. Most of these practices have been around for decades. By clustering the effective 
elements of these activities through the term “high-impact practices” we bring together elements which we know are 
effective pedagogies, without weight to any one practice over another. The vast majority of campuses can proudly point 
to multiple high-impact practices happening somewhere within their institutions, not just one or two. Thus, to talk 
singularly about high-impact practices—and the assessment of them—would be like evaluating just one tire on a car. 
To ultimately understand what makes an institution move requires looking at all of the moving parts. And given the 
intense focus across institutions of higher education on identifying, tagging, and touting their high-impact practices, 
assessment is what will separate the committed practitioners from the casual adopters. A good assessment plan for high- 
impact practices starts with acknowledging three things. One, the name alone does not make them high-impact. Two, 
evidence of effect requires assessing more than outcomes, alone. And three, assessment must be, at every stage, attentive 
to equity. Building upon these three ideas, this Occasional Paper outlines a process of effectively assessing high-impact 
practices at your institution. 
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A Comprehensive Approach to Assessment 
of High-Impact Practices 


Ashley Finley 


The great thing about effective pedagogy is that everyone wins. Students love being engaged. 
Faculty love the excitement that comes from students’ engagement. Admissions personnel 


benefit from having innovative practices to highlight. Student affairs professionals find LO 


well-deserved recognition as co-educators. Senior administrators celebrate the link 

between effective pedagogy and increased retention and graduation rates. And even ‘The biggest impediment 
though all of the above is highly simplified, there is an element of truth in each statement. _ to assessing high-impact 
practices may very well be 
the name itself. The term, 
“high-impact,” almost 
assumes efficacy. With a 


The support for these statements comes largely from scholarship around high-impact 
practices, or effective pedagogies, such as learning communities, capstones, undergraduate 
research, and community-based experiences.’ Most of these practices have been around a 
while (more like decades, than years), but the clustering or grouping of the effectiveness 
of the activities through the term, “high-impact practices,” is efficient in lumping all of | name like that, what is left 
the things we know are effective pedagogies together, without weight to any one practice —_ to assess? The answer is 

or another. And while the list will most certainly expand as we learn more about a range of plenty. 

effective pedagogies that advance student learning and success (e.g. work study, mentoring 

and advising), even as it currently exists, the vast majority of campuses can proudly point 

to multiple high-impact practices happening somewhere within their institutions, not 

just one or two. Thus, to talk singularly about high-impact practices—and the assessment 

of them—would be like evaluating just one tire on a car; to ultimately understand what 

makes an institution move, requires looking at all of the moving parts. 


The tricky thing about high-impact practices is that they can be hard to scale, hard to 
sustain, and hard to assess. An argument can be made that the first two challenges (scale 
and sustainability) are closely related to the third (assessment). Finding the administrative 
resources and political capital to reach appreciable levels of scaled practices, along with 
sustainability of resources over time, are often the result of having sufficient evidence to 
make the case that such things are worth the time and investment. A solid assessment 
plan is, therefore, crucial. But the biggest impediment to assessing high-impact practices 
may very well be the name itself. The term, “high-impact,” almost assumes efficacy. With 
a name like that, what is left to assess? The answer is plenty. And given the intense focus 
across institutions of higher education on identifying, tagging, and touting their high- 
impact practices, assessment is what will separate the committed practitioners from the 
casual adopters. 


A good assessment plan for high-impact practices starts with acknowledging three things. 
One, the name alone does not make them high-impact. Two, evidence of effect requires 
assessing more than outcomes, alone. And three, assessment must be, at every stage, 
attentive to equity. The place where scholarship has most failed our collective understanding 
of high-impact practices is in the differential access and effects on outcomes across various 


' For a background on high-impact practices, read this excerpt from the Association of American Colleges 
and Universities (AAC&U) from High-Impact Educational Practices: What They Are, Who Has Access to 
Them, and Why They Matter, by George D. Kuh (2008). Visit AAC&U’s resources page on high-impact 
practices for updated information and campus case studies. 
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student populations, and particularly those students traditionally underserved by higher 
education. There is evidence to suggest that high-impact practices are good for all students, 
though certainly not in the same ways (Finley & McNair, 2013; Kuh & O’Donnell, 
2013). The rapid and increasing diversification of American higher education demands 
equity not be an afterthought for assessment. The goal of understanding and addressing 
institutional equity gaps in student learning and success as a result of participation in 
high-impact practices must be baked into assessment plans from the beginning. 


Chucking the Checklist 


Perhaps the best and the worst thing that happened for the assessment of high-impact 
practices is that a list was created identifying the ten most prevalent experiences, including 
first-year seminars, community-based learning, senior capstones and projects (Kuh, 
2008). In 2016, ePortfolios were added to the list as an “eleventh” high-impact practice 
(Watson, Kuh, Rhodes, Penny Light, & Chen 2016). Even though a great many of these 
practices had been around for decades (see for example, Brownell & Swaner 2011), the 
value of a list was to conceptualize high-impact practices in aggregate, enabling campuses 
to recognize just how many of these experiences students might encounter during 
their college journey. The “list” has promoted campus conversations about breadth of 
high-impact offerings, how experiences might be scaffolded to encourage learning and 
development over time, and opportunities for expansion and strategic planning. 


However, the “list” can also sometimes be viewed as a “checklist.” The detriment of the 
checklist is the temptation to “check off’ which practices are being done without asking 
deeper questions about student learning, equity gaps, and even quality of delivery. This 
may be because efficacy is assumed, or because the imperative of quantity can reign over 
pleas for quality, or some other combination of factors. Regardless, when assessment is 
reduced to checking boxes, what is known is that a practice exists, rather than whether 
that practice is actually making a difference for students. 


In fairness to campus practitioners and leaders, one benefit of the checklist is to invite 
inventories for where high-impact practices exist across campus. One consequence of 
encouraging “a thousand seeds to grow” within the autonomous environments of college 
campuses is not knowing where exactly those seeds have been scattered. Doing an inventory 
(i.e., asset mapping) of where exactly high-impact practices exist is an important step for 
any assessment plan. Moreover, the best inventories should seek to pair practices with a 
description, rather than a label alone (e.g., “service-learning” or “writing intensive”), to 
be inclusive of faculty or campus divisions that may be engaging in similar practices but 
calling them by different names. 


But inventories alone cannot address critical questions about student learning outcomes, 
quality of implementation, or equity gaps. Herein lies the greatest challenge (and 
opportunity) for the assessment of high-impact practices. To do assessment well is to 
evaluate multiple components of the experience, from resources to outcomes, and to 
move beyond the checklist mentality for implementation and scaling of these practices. 


One way to go beyond the checklist in a manner that is mindful of both implementation 
and questions of practice is through employing a logic model. ‘The remainder of this 
paper applies a logic model framework to explicate how a multi-dimensional approach 
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can be undertaken for the assessment of high-impact practices. Logic models have been 


used in various contexts but follow a common structure consisting of: impact, outcomes, 
outputs, activities, resources, and assumptions (for examples, see Figure 1 and the W. K. 
Kellogg Foundation’s Logic Model Development Guide). Much has been written about 


the use of logic models to inform program evaluation, particularly in higher education. 
As a tool intended to explicate a program’s theory of change, there is little room to hide 


in a logic model. There is also every opportunity to insert commitments to equity and 
transparency at every step in the process. 


Logic Model Template 


Inputs 
[Resources] 


Processes/Activities 
[Actions, Developmental 
Programs] 


Outputs 
[Products of 
Processes/Activities] 


fim \ 


Outcomes Impact 
[Intended Effect of Outputs] [Vision for Change] 


Assumptions 


Figure 1: Logic Model Template. Adapted from W. K. Kellogg Foundation, Logic Model 


Development Guide. For more information see https://www.aacu.org/node/5682. 
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Starting with the (Very) End: Articulating the Vision 


In assessment parlance, logic models are based upon backward-design. Backward-design 
suggests that the examination of a learning experience or program should start with a clear 
sense of the intended outcomes first and then work backward to ensure the process of 
assessment is firmly rooted in the defined and operationalized objectives of the program 
or practice. But what backward design approaches can often miss, even when starting 
with the outcomes, is the imperative for doing the work in the first place. The question 
that should come before “What are our outcomes?” is “What is our commitment to 
or vision for student learning and success?” Thus, the starting point with logic models 
is not necessarily a list of outcomes (those come next). Rather, the starting point is a 
statement of the intended value, purpose, and worth of the endeavor. Effective vision 
statements (sometimes referred to as “impact statements”), particularly with regard to 
high-impact practices, should situate the high-impact practice within a larger imperative 
or instiutitonal priority for student learning and success. This statement is also an 
opportunity to highlight an institution’s commitment to or strategic vision for equity 
and inclusive excellence. For example, “The university strives to provide a transformative 
and equitable education for all students through innovations in teaching, learning, and 
community engagement.” 


Vision statements may be drawn from a college or university mission statement or 
strategic plan. They are not intended to be directly assessable but rather to serve as a 
kind of “North Star” for guiding the work and providing inspiration throughout the 
process. Additionally, because logic models are meant to serve as communication tools, 
vision statements can often be helpful for reminding people of the imperative of the 
work, or for inviting new colleagues into a shared understanding of that imperative. 
When colleagues groan, “What’s the point of these learning communities?” The vision 
statement provides a window into why the effort of that specific high-impact practice 
is part of a grander vision for student learning and success. Initiative fatigue is real for 
faculty. Having language that inspires and energizes stakeholders who are investing time 
and energy into the process of implementing high-impact practices can provide clarity on 
intentions, roles, and engagement. 


Like every part of the logic model, vision statements should be examined over time 
for revision and/or updating. In other words, they should be re-assessed. This can be 
particularly important as commitments to equity emerge or change by becoming more 
targeted, better articulated, more fully understood, or all of the above to improve collective 
understanding. In this way, the “Vision” box of logic models can also help capture the 
ways in which campus mindsets and values may shift and evolve around the use of high- 
impact practices (see Appendix). 


Establishing Endpoints and Timelines: Articulating Outcomes 


Once the imperative for high-impact practices has been articulated, the next step 
is to specify assessable outcomes that are linked to the expected changes in students’ 
development and learning as a result of their participation in the experience. Outcomes 
within logic models, however, go a step further to distinguish between outcomes that 
are short-term, intermediate, and long-term. Because outcomes do not occur at exactly 
the same time, it is helpful to specify how much or for how long evidence would need 
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to be gathered to indicate changes in a desired outcome. The guiding questions for this 


section of the logic model should address changes in student learning and success, and 
also effects on equity gaps. For example: What changes are expected with regard to short- 
term, intermediate, and long-term student learning and success outcomes as a result of 
students’ engagement in one or multiple high-impact practices? In what areas, do we 
expect equity gaps to be reduced over time? What learning will be acquired or reinforced 
from this experience? How does the experience connect, relate to, or integrate with 
learning from other curricular and co-curricular learning experiences? 


Short-term outcomes often include outcomes pertaining to students’ learning and 
development because changes in these outcomes can typically be assessed in a relatively 
short amount of time (i.e., a semester or a year). AAC&U’s Essential Learning Outcomes 
(www.aacu.org) may help to inform a list of potential outcomes, many of which have been 
widely adopted across two- and four-year colleges and universities. Additionally, program 
learning outcomes and course learning outcomes are also an excellent starting point for 
connecting outcomes with high-impact practice experiences. Campuses would be wise to 
connect these outcomes with supporting employer research from AAC&U (see https:// 
www.aacu.org/public-opinion-research) and other sources (see, for example, Carnvale 
& Smith, 2013), to make explicit the link between learning and career preparedness 
for students. Short-term outcomes might also reflect campus priorities for improving 
students development of intrapersonal and interpersonal competences, such as resilience, 
coping with failure, self-efficacy, belonging and relationship-building, to which high- 


impact practices may uniquely contribute (Finley, forthcoming). 


But tracking short-term improvements or boosts in learning and development outcomes 
for students will not tell the whole story about high-impact practices. For one, campus 
stakeholders tend to be interested in gains over time with regard to learning and 
development outcomes, not just at a single point in time. Stakeholders may also want to 
assess cumulative changes in learning and development across students’ engagement in 
multiple high-impact practices. Additionally, leadership at all levels will want to consider 
effects of high-impact practices on retention, persistence, and graduation rates. Because 
such aims necessitate gathering evidence over a longer period of semesters and years, 
these outcomes are typically considered to be “intermediate” to recognize the need for 
accumulated or tracked data before viable results will emerge. 


Long-term outcomes can take many years to gather sufficient evidence to indicate 
progress. When thinking about high-impact practices, long-term outcomes might be 
thought of as cultural shifts in practices, ideologies, or goals for reaching increased levels 
of scale. Moving from a teaching-centered to student-centered campus culture; becoming 
more equity-oriented; embedding high-impact practices throughout general education or 
within most academic programs are a few examples. These outcomes may also involve the 
assessment of long-term trends in student learning as a result of engagement in sustained 
high-impact practices, such as across cohorts and/or comparative gains across class ranks 
or credit accumulation (e.g., first-year to senior comparisons). 


Whether the outcome is short-term, intermediate, or long-term, considerations of equity 
should always be included. Without these questions, campuses ignore one of the most 
persistent findings about student success in higher education—that student experiences 
are far more different within campuses than across them. The ability to locate where and 


Considerations of equity 
should always be included 
in your outcomes. 
Without it, campuses 
ignore one of the most 
persistent findings about 
student success in higher 
education—that student 
experiences are far more 
different within campuses 
than across them. 
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how experiences differ demands the articulation of outcomes, at multiple levels, that 
recognize students are not a homogenous group. Examining equity within high-impact 
practices across these categories of outcomes might involve, for example, the assessment 
of differences in learning gains across student populations (short-term), comparative 
analyses of retention, persistence, and graduation rates (intermediate), and accumulated 
evidence that couples (hypothesized) reductions in equity gaps over time with climate 
survey data on broader attitudinal changes regarding diversity and inclusion across 
student populations, including minoritized groups (long-term). 


Good (Enough) Evidence: Articulating Outputs 


Outcomes require evidence. It is an obvious enough point until one comes to grips 
with the vast amounts of evidence on most college or university campuses and all the 
possibilities for gathering more. There is indirect evidence (e.g., student surveys, focus 
groups) and direct evidence (e.g., student work products graded or scored using rubrics, 
standardized assessments scored based upon students’ written responses). There is the 
evidence collected in classrooms (graded student work and course evaluations) and 
institutional surveys on the student experience (e.g., NSSE, CIRP). There is also evidence 
collected after students leave (i.e., alumni surveys). The degree to which any of it matters 
for high-impact practices is one to be thoughtfully considered and then connected to the 
intended learning outcomes for a particular high-impact practice. 


When this is done, what often comes to light is how little we know about the connection 
of high-impact practices with direct evidence of students’ demonstrated learning. For 
over a decade, the evidentiary case for high-impact practices, singularly or in aggregate, 
has largely been made through indirect sources, such as the National Survey of Student 
Engagement (NSSE). This is especially concerning as the emphasis on linking high-impact 
practices with broad, transferable learning outcomes, particularly the kind connected with 
career preparedness, is only likely to grow. Additionally, because high-impact practices 
have been shown to have differential effects on traditionally underserved students’ self- 
reported gains and also on retention and graduation rates (see Finley & McNair, 2013; 
Finley & McNair, 2016; Kuh, 2008; Kuh & McDonnell, 2013) it is critical to have direct 
evidence from different populations of students to help tease out actual learning gains 
that may be correlated with other findings. 


While indirect outputs might include evidence from national or local campus surveys, 
direct outputs can reflect specific products of student work intended to address particular 
learning outcomes. Examples of such work products might be: signature assignments 
produced in capstone courses, artifacts contained in ePortfolios or the ePortfolio itself, 
reflections or group projects produced in first-year seminars or first-year experiences; 


undergraduate research papers or presentations. All of these types of work products can 
be matched with rubrics intended to assess the desired learning outcomes, ideally over 
time to gauge progression in learning. The AAC&U VALUE rubrics provide a template 
for these types of rubrics across a range of learning outcomes (www.aacu.org/value). 


‘There is no such thing as a perfect survey, rubric, or focus group script. This is why evidence 
is often best when used in combination with a variety of sources and methodologies. 
Gathering enough substantiating evidence to indicate high-impact practices are meeting 
the intended outcomes is not an endeavor that should seek, or even requires, perfection. 
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It is a process that should target improvement of practices and promote conversations 
among stakeholders focused on evidence-based decision-making. When focused on 
institutional advancement, the best outputs for high-impact practices will strike a balance 
between indirect and direct sources of evidence to ensure no one source of data (and its 
inherent methodological caveats) overly guide decision-making. Multiple data sources 
will also ensure multiple opportunities for disaggregating data (of any type) across student 
populations to assess for equity gaps and related strategies for improvement. 


How are we implementing 


‘The “Activities” box of logic models can often seem like the easiest piece to fill in when this experience such that 
thinking about high-impact practices. When the question is: “What are we doing”, the _ it will produce good 
easy response is to list high-impact practices by name (e.g., capstone, service-learning, evidence of outcomes?” 
learning communities). But the real question 4 the “Activities” box is, “How are ye When framed this 
implementing this experience such that it wi produce good evidence (outputs) of way, the assessment of 
outcomes?” When framed this way, the assessment of activities related to high-impact ee : 

, ; . Sonate activities related to high- 
practices moves from a focus on what is happening, to a focus on how well it is being  * ; 
done. impact practices moves 

from a focus on what is 

‘The notion that experiences are only high-impact when implemented with attention to happening, to a focus on 
quality is a familiar refrain within research and discussions of these practices (Clayton- — fpgyy wel] it is being done. 
Pedersen & Finley, 2010; Finley & McNair, 2016; Kuh & O’Donnell, 2013). Kuh and 
O'Donnell (2013), for example, laid out a helpful list of eight quality dimensions that 
often accompany high-impact practices (p 10). These quality dimensions are: 


The Quality of Engagement: Articulating Activities 


1. Performance expectations set at appropriately high levels; 
2. Significant investment of time and effort by students over an extended period 
of time; 


ra 


Interactions with faculty and peers about substantive matters; 

4, Experiences with diversity, wherein students are exposed to and must contend 
with people and circumstances that differ from those with which students are 
familiar; 

5. Frequent, timely and constructive feedback; 

Periodic, structured opportunities to reflect and integrate learning; 

7. Opportunities to discover relevance of learning through real-world 
applications; and 

8. Public demonstration of competence. 


an 


Assessing high-impact practices in terms of the quality with which they are being 
implemented mean, at minimum, addressing the degree to which these elements exist. 
For example, in what ways do syllabi and assignments provide high expectations for 
student learning and engagement in their experience? In what ways are students investing 
significant time and effort throughout the experience, whether over a semester, a year, 
or longer? How are students encouraged to explore diverse perspectives and to engage 
with people different from themselves as part of the experience? Are there consistent 
opportunities for reflection and constructive feedback? Will students have the opportunity 
to present their work or “teach out” what they have learned? 
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Using a rubric, for example, that connects quality dimensions with levels of intensity 
(e.g. high, medium, low), syllabi and assignments can provide one source of information 
on the quality of high-impact practices by serving as artifacts to be scored. Faculty can 
also use such a rubric to self-assess their own delivery of the experience. Additionally, 
end-of-experience student surveys can provide insights into the degree to which students 
perceive the quality dimensions existed as components of the experience and which 
components made a difference in their engagement. Combining these forms of data, 
direct (i.e., syllabi, assignments) and indirect survey responses and self-assessments, can 
inform the interpretation of evidence on outcomes. For example, if evidence on actual 
learning gains is weak or unclear, the assessment of the quality of components of the high- 
impact practice can expose areas of implementation that can be strengthened or adjusted. 


Quality dimensions also hold particular significance in thinking about equity. Each 
dimension is an opportunity to consider how culturally responsive teaching can shape 
student’s engagement in the experience, whether inside or outside the classroom. 
Lenses of cultural competence and critical consciousness can be employed to encourage 
students, particularly those from underserved backgrounds, to engage in a high-impact 
practice through their own vantage point and to value the perspective they bring (Larke, 
2013). For example, as interaction, reflection, feedback, and even demonstrations of 
competence are incorporated into the experience, each should consider a diversity of 
students’ perspectives, backgrounds, and cultural touchstones. Students should also be 
invited to engage in quality dimensions through the vantage point of cultural wealth— 
that each student uniquely brings valuable cultural resources to the learning experience 
that inform their understanding and application of the learning (see for example, Yosso, 
2005; Rendén, Nora, & Kanagala, 2014). 


Thinking Boldly About Resources: Articulating Inputs 


High-impact practices are often perceived as requiring a lot of resources, though arguments 
have also been made for budget-friendly solutions (Wellman & Brusi 2013). Regardless, 
of the type of high-impact practice (or any pedagogical approach for that matter), there is 
no question that resources matter. Nevertheless, despite the temptation to jump directly 
to available resources, the use of logic models can help in resisting this approach. By 
working backwards from the vision, through intended outcomes, to needed outputs, and 
intentional activities, inputs should come last to allow the process of how high-impact 
practice(s) should operate to be fully articulated first. This ensures that understanding 
what is needed to execute the process is not artificially constrained by a scarcity mindset 
(“We don’t have this or that.”). Instead, what is needed for execution is informed by the 
articulated elements of the plan itself. 


In the strictest sense, inputs should be associated with what is needed for intended 
activities. But in the best sense, and in the case of high-impact practices, they should be 
thought of expansively. First, inputs should account for the specific high-impact practices 
currently operating on campus. When programs like first-year seminars, learning 
communities, service-learning, internships, and the like, already exist, they are resources 
because they have budgets and people connected to them. It may not be enough money 
or enough people, but even meager resources count. And the resource of knowledgeable 
staff who can teach others should not be underestimated. The process of creating an 
inventory of existing high-impact practices on campus can serve as a valuable step in 
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understanding the extent of available resources. Such an inventory should also be paired 


with an examination of levels of participation in those experiences across populations of 
students, specifically those traditionally underserved by higher education, to understand 
access and participation gaps as part of an overall commitment to addressing equity. 


Second, if the type or extent of practices is unknown, asset-mapping is a helpful way to 
get a handle on where high-impact practices are occurring. For example, department 
chairs can be a helpful group to query about the types and degrees with which high- (a 
impact practices are occurring within programs. When conducting asset-mapping it can 
be helpful to describe practices, rather than asking stakeholders to respond to a particular 
label or name. For example, not everyone knows what constitutes service-learning or even responsible for delivering 
community engagement. So, defining what a practice is and leaving it up to respondentsas high-quality, high-impact 
to what it is called can go a long way toward identifying the range of practices happening practices. Almost by 

across campus and toward building common language over time. 


Faculty alone are not 


definition, high-impact 


Third, when considering who or what types of programming might contribute to the practices extend beyond 


activities associated with high-quality, high-impact practices, there is an opportunity to the classroom in ways 
think expansively across campus offices and services. Faculty are resources, as are student that can actively and 

affairs professionals, librarians, professional advisors, peer mentors, orientation leaders, intentionally involve a 
and community members (and the list goes on). Centers for teaching and learning, for range of campus- and 


community engagement, for research, for diversity and multiculturalism, and for writing community-based 


all provide a host of services that can be tapped to support faculty professional development See ee 


and student development. Offices of Institutional Research, of Effectiveness, and of 
Assessment can all contribute vital assessment assistance for telling a story of efficacy and 
of equity regarding high-impact practices on campuses. Thus, the approach to assessing 
inputs for high-impact practices is to evaluate just how much exists before determining 
what is still needed. Along with this comes the recognition that faculty alone are not 
responsible for delivering high-quality, high-impact practices. Almost by definition, high- 
impact practices extend beyond the classroom in ways that can actively and intentionally 
involve a range of campus- and community-based resources. 


Finally, campuses are sometimes seeking to develop new high-impact practices. The 
compelling research on ePortfolios (Watson et. al., 2016; Eynon & Gambino, 2018; 
Eynon & Gambino, 2017), for example, has encouraged many campuses to explore 
implementing this high-impact practice into some, if not all, of the student experience. 
In this case, resources for beginning ePortfolios would become inputs and the activities 
related to launching or piloting ePortfolios would be placed in the “Activities” box (e.g., 
faculty development, new student orientation, curricular connections). But it is worth 
pointing out that well before these boxes are completed, it is assumed that there has 
already been an articulation of how ePortfolios contribute to an imperative for student 
learning, equity, and success; relevant outcomes; and sources of evidence (as preceding 
boxes in the logic model). Even though inputs can be thought of as the gas that makes the 
car go, one still needs to know where the car is going in the first place. 


Final Considerations for a Comprehensive Approach 


A final point on logic models as a tool to assist in assessing learning within high-impact 


practices is that assumptions underlie every stage of developing the model and every box 
that gets filled. Because logic models are intended to be collaborative tools, there is the 
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opportunity for colleagues to respectfully challenge each others’ assumptions, and then 
to test the assumptions that remain. Common assumptions often revolve around the 
belief that everyone is on the same page about mission and goals; that faculty will or 
will not engage (and just who or what those faculty or departments are); that evidence 
does or does not exist and who has it; or that resources are scarce. In addition, there are 
assumptions about how students learn, where they learn, and how to document that 
learning that need to be explored. 


Undertaking the process of developing logic models helps to address perhaps the biggest 
assumptions ofall: that high-impact practices cannot be assessed or are too difficult to assess. 
The mistake that is often made in this assumption is thinking phenomena as complex as 
student engagement and student learning and development can be neatly tied to a single 
act of assessment. That is an impossible task. Honoring the multi-faceted nature of what 
constitutes well-aligned (think: mission and outcomes), well-mined (think: evidence), and 
well-designed (think: high-quality) high-impact experiences requires fortitude, process, 
and logic. It also requires, as the preceding argument hopefully conveys, an unwaivering 
commitment to equity—not just in disaggregating outcomes data—but in every step of 
assessment along the way. 


Just as a logic model approach to assessing high-impact practices can assist campuses in 
taking a comprehensive approach to gauging the institutional efficacy of these practices, 
such an approach can also contribute significantly to a broader collective understanding 
of the value of these practices nationally. This type of multi-dimensional approach would 
help to address a deep well of questions about quality, equity, and learning that arise 
at multiple points across the processes of implementation and development of these 
practices. We know that high-impact practices (when done well) empower students. It 
is time to empower ourselves—as practitioners, researchers, and leaders—to fully assess 
their worth. 
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Appendix: Sample Logic Model for Assessing High-Impact Practices 


Inputs Processes/Activities ~~ Outputs 7 Outcomes 
[Resources] [Actions, Developmental [Products of Processes/ [Intended Effect of Outputs] [Vision for Change] 
Programs] Activities; Sources of 


e Inventory of existing ” Evidence] Short-Term (evidence collection 
High-impact practices: Qualities of High-Impact approximately one semester to 


o Learning Communities Practices: Direct Evidence one year) 
o First-year © Performance expectations set e Use of rubrics (e.g. VALUE © Semester or one-year gains in 
Seminars/ Experience rubrics) to score student learning outcomes such as 


at appropriately high levels; 


o Community oot ; ; work products, such as: critical thinking, written 
Engagement (e.g. PSI SCant ny eStore o Group projects communication, civic skills “The university 
service-learning; global and effort by students over an oO Reflection journals (such as civic awareness or strives to provide a 
learning; community- extended period of time; o Assessment of ePortfolios perspective-taking) transformative and 
based research; e Interactions with faculty and oO Signature work e Disageregated gains by equitable education 
clinicals and field peers about substantive fo) Capstone projects student populations for all students 
work; internships) o Community-based Intermedi iden through innovations 
o ePortfolios sca projects ection approximately 2- in teaching, learning, 
i Taeetataaal e Experiences with diversity; © Oral presentations years) and community 
Research/ Office of e Frequent, timely and e Institutional Research data e Improvements in retention engagement.” 
Assessment constructive feedback; a post-graduation e Changes in retention by 
¢ Teaching and Learning @ Periodic, structured nee student populations 
ae opportunities to reflect and indeee Boden e Intermediate gains 
© Office of Community- integrate learning: = Long-Term (eviden lection 
Engagement “ 7 e Institutional Research requites at least five or more 
© Office of Diversity and ° pe aaa Laman : Surveys related to student Itiple cohorts) 
Inclusion ee pare teaening (Ce. e Changes in graduation rates 
a Cortnnnity Partners real-world applications; and National Survey of Student sae 
e Public demonstration of Engagement/Community 
e Curricular Supports and competence. College Survey of Student e Improvements in student 
Budgets (e.g. general Engagement; CIRP; Climate learning over time 
education, departments) (fom Kuh & O’Donnell, 2013, Surveys) e Persistent reductions in 
© Student Affairs/Student Ensuring Quality and Taking High- ¢ Locally developed surveys equity Gaps cvet time 
Life Impact Practices to Scale, e Focus groups ¢ Changes in participation rates 
e Alumni/Career Services Washington, DC: Association of ® Alam survevs within and across high-impact 


practices across student 


e Advising American Colleges and 
populations 


Universities, p. 10) 


e Course evaluations 


The university assumes that the more students engage in meaningful learning experiences, supported by aligned assessments that allow multiple means to demonstrate 
competence, that students will feel valued and mote likely to persist, exhibit resilience, and experience success in future employment. In order to ensure meaningful learning 
experiences are available to all students, the entire university community needs to be involved in implementation, including students themselves. 
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